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Abstract 

The fcth projection function Vk{K, •) of a convex body K C d > 3, is a 
function on the Grassmannian G(d, k) which measures the fc-dimensional 
volume of the projection of K onto members of G{d,k). For k = 1 and 
k = d—1, simple formulas for the projection functions exist. In particular, 
Vd-i{K, •) can be written as a spherical integral with respect to the surface 
area measure of K. Here, we generalize this result and prove two integral 
representations for Vk{K, ■),k = 1,... ,d—l, over flag manifolds. Whereas 
the first representation generalizes a result of Ambartzumian (1987), but 
uses a flag measure which is not continuous in K, the second representa¬ 
tion is related to a recent flag formula for mixed volumes by Hug, Rataj 
and Weil (2013) and depends continuously on K. 
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1 Introduction 

Let /C be the space of convex bodies (non-empty compact convex sets) in d > 
3, supplied with the Hausdorff metric. For K G K. and k G {0, ....,d}, the 
fcth projection function Vk{K,-) of iF is a continuous function on G{d,k), the 
Grassmannian of fc-dimensional subspaces in K^, and is defined by 

Vk{K,E) = Vk{K\E), EGG{d,k). 

Here K\E is the orthogonal projection of K onto E and VJ denotes the fcth 
intrinsic volume which, for a body in the fc-dimensional space E, equals the (fc- 
dimensional) volume in E. The projection functions vo{K, •) = 1 and Vd{K, ■) = 
Vd{K) are trivial. For fc = 1, vi{K\E) is the width of K in direction of the line 
E, hence 

vi{K, E) = h{K, u) + h{K, —u), 

where h{K, •) denotes the support function of K and u is in the direction of the 
line E. 
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For k = d — 1, we have a simple and well-known integral representation, 
namely 

Vd-i{K,x^) = ]- [ \{x,u)\Sd-iiK,du), x€S'^~^. (1.1) 

Z Jgd-l 

Here, S‘^~^ is the unit sphere, (•,•) denotes the standard scalar product in 
and Sd-i{K,-) is the {d — l)st surface area measure of K (see [17], for the 
standard notions from convex geometry which we use). For bodies K G }C 
which are centrally symmetric and smooth enough, a corresponding integral 
representation for arbitrary projection functions is known. This involves the 
projection generating measure pk[K,-) of iF, a (signed) measure on G{d,k), 
and reads 

Vk{K,E) = ^ [ \{E,F)\pk{K,dF), EGGid,k), (1.2) 

JGid,k) 

where \{E,F)\ is the absolute determinant of the projection of E onto F. (II.2|) 
holds more generally for generalized zonoids K , but it is also known that a 
corresponding formula for all centrally symmetric bodies K cannot hold, for 
k G {1,..., d — 2}, at least not with a (signed) measure Pk{K^ •) on G{d, k), (see 
[^ and the remarks on p. 635 of |12)1. 

Let V C K. he the dense subset of convex polytopes. For fc = 1, d = 3 
and P G V, Ambartzumian introduced a new concept by showing that 

the width function vi{P,-) of P has an integral representation with a certain 
measure on a flag manifold. It is a hrst goal of this paper to generalize this 
result to arbitrary dimensions d and k. To be more precise, we introduce, for 
1 < g < d, the flag manifold 

F{d, q) := {{u,U):U G G{d, q),uGUn 

With the usual topology, F(d, q) is a compact space. For a polytope P and 
k G {1,..., d — 1}, we define a Borel measure Tk{P, ■) on F{d,d— k) by 

rfc(P,.):= ^ 14(F) [ l{{u,F^)G-}n‘^-^-’^{du). (1.3) 

FeJ^k(P) Jn{P,F) 


Here, iFk{P) is the collection of fc-faces of P, n{P,F) is the intersection of 
the normal cone N{P,F) of P at F with the unit sphere S‘^~^ (that is, a 
(d — 1 — /c)-dimensional spherical polytope), F-*- G G{d,d — k) is the linear 
subspace orthogonal to F and is the Hausdorff measure of dimension 

d — 1 — k. For d = 3, the measure ti(F, •) coincides, up to the factor 27r, with 
the one introduced by Ambartzumian ElE]. 

Theorem 1.1. For 1 < k < d — 2 and P G P, we have 


Vk{P,E) 


= UJ 


-1 

l-k 


{u^ n H, M-*- n F-*-)^ 


F{d,d-k) 
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Tk{P,d{u,U)), EGG{d,k). 

(1.4) 



Here, ujn denotes the surface area of the n-dimensional unit ball and the 
integrand is defined as 0 for u G E. This result remains true for k = d — 1 and 
reproduces in the special case of polytopes equation (EU, if the integrand is 
properly interpreted as ||M|i?"*-|| = |(m, u)| for a unit vector v G E-^. 

For d = 3 and k = 1, Ambartzumian extended this integral formula for 
projection functions to arbitrary convex bodies AT C by using polytopal 
approximation. Now, let d > 3 and 1 < k < d — 2. With a given convex body 
K G 1C, we can associate an approximating sequence of polytopes Pi —>■ K, 
Pi G V. Since Tk{Pi,P{d,d- k)) = u}d-kVk{Pi) and Vk{Pi) Vk{K), we can 
choose a weakly convergent subsequence of the measures Tk{Pi,-), i G N, to 
obtain a limit measure Tk{K,-). However, it is important to notice that even 
for d = 3 and k = 1, the integrand in (11.41) is not a continuous function. This 
indicates that the extension of Theorem 11.11 to arbitrary convex bodies K by 
a limit procedure requires further arguments (see Proposition 13.2p . Moreover, 
as was shown by an example in uni, the limit measure Tk{K, ■) depends on the 
approximating sequence (Pijigfsj and is, thus, not continuous in K. 

The second and major goal of this work is therefore concerned with an 
integral representation of Vk{K, •), in the spirit of (jl.4[) . but with a flag measure 
ipk{K,-) on F[d,d — k), which depends continuously on K G 1C. To be more 
precise, let tpkiP, ■), for P GP,he the measure on F{d, d — k) given by 

MPg)--= E / / !{(«,[/) G-} 

JG({u),d—k) 

X {U,Fp^ iy^%idU)n‘^-^-\du). (1.5) 

Here, G{{u),d— k) denotes the Grassmannian of {d— fc)-flats containing the line 
(u) generated by u and corresponding invariant probability measure. 

As follows from results in [21HH], '4’k{P, •) satisfies a local Steiner formula for P 
and therefore has a continuous extension to all K G 1C. Our main result is the 
following. 

Theorem 1.2. For 1 < k < d— 1, there is a function g on G(d, k) x F{d,d— k) 
such that 

Vk{K,E)= f g{E,u,U) 'ifk{.K,d{u,U)) (1.6) 

JF{d,d-k) 

holds for all K G 1C, and almost all E G G(d, k). 

Here ‘almost all’ refers to the invariant probability measure on G{d, k) 
and the ‘exceptional set’ may depend on K. 

The setup of the paper is as follows. In the next section, we introduce the 
necessary notation and we collect some background information. In Section 3, 
we give the proof of Theorem 11.11 The subsequent two sections are concerned 
with the proof of Theorem 11.21 The final section discusses some functional 
analytic aspects concerning an integral equation which connects the two repre¬ 
sentations and dUD. In the Appendix we prove an auxiliary lemma and 
a combinatorial identity, both of which are used in the solution of this integral 
equation. 
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2 Preliminaries 


In the following, we work in Euclidean space > 3, with scalar product 

(•, •) and norm || • ||. We denote the unit ball and the unit sphere by B‘^ and 
respectively, and let W be the j-dimensional Hausdorff measure. We use 
the abbreviations Kd '■= and ojd ■= dud- For a set A C bdA is 

the topological boundary and lin A is the linear hull. For real numbers a, b, we 
denote the minimum by aA&. We also use the symbol A for the exterior product 
of vectors; the usage will always be clear from the context. We have already 
introduced the classes fC and V of convex bodies and convex polytopes. For 
P G V, ^k{P) is the set of /c-dimensional faces of P, fc S {0,..., d — 1}. For 
P GV and a face F G lJfe=o ^k{P) of P, N{P, F) is the normal cone of P at F 
andn(P,P) := 7V(P, P) n 5"^"^ 

The Grassmann manifold G((i, k) is the set of fc-dimensional linear subspaces 
of supplied with its usual topology and with the invariant (probability) 
measure For subspaces A,Bg G{d, fc), |(A, B) \ is the (absolute value of the) 
determinant of the orthogonal projection of A onto B (or vice versa) and {A, B)^ 
is the corresponding square value. We shall repeatedly use that \{A,B)\ = 
B^)\ for A,B G G{d, k). More generally, we need the ith product {A, B)i, 
i = 0,... ,k A {d — k), for which we refer the reader to [1^, for a detailed 
description. Roughly speaking, {A, B)i is the length of the orthogonal projection 
of a simple unit fc-vector representing B onto the space of exterior products of 
unit simple (fc — i)-vectors in A with corresponding i-vectors in the orthogonal 
complement A^. We mention that (A, B)i = (P, A)i and emphasize the special 
case {A,B)o = \{A,B)\. For a subspace L G G{d,k) and j G {0,...,d — 1}, 
we write G{L,j) for the manifold of subspaces M G G{d,j) which contain L, 
if j > k, or are contained in L, if j < k. We also denote the corresponding 
invariant probability measure on G{L,j) by 

Measures on flag manifolds have been investigated in recent years for con¬ 
vex bodies and more general sets (for example, sets of positive reach). Since 
we are interested in projection properties, the concentration on convex sets 
seems natural. For convex bodies K, flag measures can be introduced in var¬ 
ious ways, as projection means of support measures, by direct representations 
on the normal bundle, or by a local Steiner formula, generalizing the classical 
two local descriptors of convex sets, the area and curvature measures. A survey 
on flag measures with further historical remarks and references is given in [14] . 
Here, we only need area-type flag measures, as they were studied and used in 
m and m- For 1 < q < d, we consider the flag manifold F{d,q), defined 
in the introduction and consisting of pairs (u,U) of a subspace U G G{d,q) 
and a unit vector u in U. For pG{0,...,d — 1}, we shall also use the manifold 
F^{d,p) := {(m, V) G 5"^“^ X G{d,p) : u G V^}, which arises from F{d, q) by the 
orthogonality map {u, U) ^ {u, C/-*-) (thus p = d — q). Another simple transfor¬ 
mation replaces F^{d,p) by F^{d, d—l—p) via the map (u, U) >->■ (u, U-^du^). 

The two (series of) flag measures which we introduced in the previous sec¬ 
tion are defined on F{d, d — k),k G {1,..., d — 1}. The first, Tk{K, •), is given, 
for polytopes AT, by (11.31) and leads to the projection formula in Theorem ll.il 
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Although this flag measure and the projection formula can be extended to ar¬ 
bitrary bodies if S /C by a compactness argument, the extended measure is not 
unique and thus is not continuous in K. Details will be given in the next section. 
It therefore seems that this flag measure is less interesting for applications in 
convex geometry. The second measure, il^kiK, •), was also defined for polytopes 
in a direct way, by (EH, but it has a (weakly) continuous extension to all con¬ 
vex bodies if G /C, as follows from the subsequent local Steiner formula (see 
[Tni[T3]). For if S /C, j G {0,... ,d— l },/9 > 0, and a Borel set rj C F^{d,j), we 
consider the local parallel set 

MW) (if, 77 ) := {E G A(d, j) : 0 < d(if, E) < p, (^(if, A), T(A)) G r,} . 

Here, A{d,j) is the affine Grassmannian, d{K,E) the (Euclidean) distance be¬ 
tween if and E, u{K,E) the direction from if to E in which the distance is 
attained, and L{E) G G{d,j) the subspace parallel to E G A{d,j)- Notice that 
the direction u{K, E) is unique, although it may be realized in more than one 
pair {x,y) € K x E. Then, Mp^\K,rf) is a Borel set in A{d,j) and its Haar 
measure p1j{Mp^\K,'q)) has a polynomial expansion in p, 

p^^(M«(if,7y)) = ^ E 

m—Q ^ ^ 


with coefficients Sm\K, •) which are finite (nonnegative) measures on F-^{d,j). 
There are many different ways to formulate this local flag-type Steiner formula 
and to normalize the measures. Here, we based the result on the normaliza¬ 
tion of the Haar measure used in m- The flag measure Sm\K,-) has the 
property that its image under the mapping {u, H) 1 —>■ u is a multiple of the mth 
area measure Sm(,K, •) of if, and the total measure is proportional to the mth 
intrinsic volume of if. More precisely, 

5«(if,F^(d,j)) = 

For a survey on the various flag measures and their interrelations, see |14j . 

The measure ijjk{K,-), defined for polytopes by (11.51) . is closely related to 
the measure In fact, in [TUI p. 21 ] it is shown that 


L 


FX{d,d-l-k) 


fiu,V+{u))si‘^-^-'^Hp,diu,V)) 


Wfe+i 

Wd 


lF{d,d-k) 


f{u,U) ipk[P,d{u,U)), 


( 2 . 1 ) 


for all measurable functions / : F{d,d — k) —>■ [0,oo). Since ^ 
depends continuously on if G /C, m shows that tpkiK,-) has a continuous 
extension to all convex bodies if and that 

^p{K,Fid,d- k)) = ^Vk{K). 

{ k) 
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Moreover, (12.11) then holds for general bodies K G 1C. 

For our proof of Theorem 11.21 we use measure geometric representations of 
the intrinsic volumes and the flag measures, described in more detail in | 12 j . 
Namely, for a convex body K G 1C, let 

HOT K := {{x,u) GhdK x S‘^~^ : {u,y — x) < 0 for all y G K} 


be its unit normal bundle. It is known that at H'^“^-almost all {x,u) G noiK, 
there exist principal directions ai{K;x,u) G S‘^~^ and associated principal cur¬ 
vatures ki{K-,x, u) G [0, oo], z = 1,..., d — 1, and we put 


Kj{K;x, u) 


U^eIk^{K;x,u) 
nf=i \/l + h{K-,x,uY 


and 

Aj{K; x,u) := Vai{ai{K]x,u) : i G /}, 

whenever I C {1,..., d—1} (with the appropriate interpretation, if ki{K] x, u) = 
oo for some i G {1,..., d — 1}, or if J = 0, cf. [12]). Then, the fcth intrinsic 
volume of K can be written as the integral 


VkiK) = ^ [ V Ki{K;x,u)H‘^-\d{x,u)), (2.2) 

Wd-fe JnorK 


where the sum is taken over all subsets I of {l,...,d — 1} of given cardinality, 
cf. [121 p. 637]. Moreover, in [12], (generalized) flag measures Tk{K-,-) on x 
F'*~(d, d— 1 —fc) and their image measures Clk{K; •) = rfc(it'; x •) on F^{d, d— 
1 — k) were introduced and it was proved, for measurable functions g : x 

F-*-(d, d — 1 — /c) —>■ [0, oo), that 

fg{x,u,V)Tk{K;d{x,u,V)) = ^ ^ ^ ^ Ki{K;x,u) (2.3) 

J dnoriC 

X [ g{x,u,V){V,Ai{K-,x,u)f v'^ti_k{dV)'H‘^~^{d{x,u)). 

J G{ud^ ^d—l — k) 

It follows from [TU (12) and (25)] and [TU p. 639] that the two measures 
and Clk{K;-) are proportional (they both are averages of the 
same area measure of projections). Since 

TkiKiR^^ X F^(d, d - 1 - fc)) = ClkiK; F^{d, d - 1 - fc)) = I4(iF) 

(this follows from [121 P- 641]) and 

MK,F{d,d-k)) = J^s]^-^-‘^\K,F^{d,d-l - fc)) = !^Vk{K), 

^k+i 

we therefore get 

Clk{K-,-)= ^ (2.4) 

Xld-k^k+l 

which yields a corresponding measure geometric formula for ipkiK, •). 


6 









Proposition 2.1. We have 


f g{u,U)^k{K,d{u,U)) = f ^ ]K/c(ii:;a;,w) 
J J nor K \ T\ 1 


\i\=k 

X [ g{u,V^){V,Ai{K-,x,u)fvl^{dV)'H‘^~^{d{x,u)), (2.5) 

JG{u^,k) 

for all continuous functions g on F{d, d — k), where F = {1, ... ,d — 1} \ I. 
Proof. From (12.ip . (12.31) and (1^ . we obtain 

J9{u,U) ijjk{K, d{u,U)) 

J diu, W + (u)) nk(K; d(u, IF)) 


(V) 


Kj(K;x,'i 

|Z|=d-l-/c 


> G{u-^ ,d—l — k) 


g{u,W + (u)) 


X (iF, Aj{K; X, u)f u^_,_kidW) %‘^-\d{x, u)) 


[ X! ^i{K;x,u) 

J nor K \T\ J 1 I 


-L ..-L 


f G{u^ ,d-l-k) 


g{u,{W^Au^)^) 


X {W'^ ,Aic[K]x,u))^ v'f[_i_^{dW)'H^ ^{d{x,u)) 


' nor K 


Kj{K;x,'i 


|/|=d-i-fe 


IG{ur,k) 


g{u, v-‘ 


X (V,Aic(K;x,u)}^iy^ {dV)n^-\d{x,u)) 


cK 


y^ K/c(i^; X, 1 


lG(ur,k) 


g{u,v^ 


X {V, Ai{K;x,u))‘^ vf: {dV)'H‘^ ^{d{x,u)), 
which proves the assertion. 


□ 


3 Proof of Theorem 11.11 

We start with a slightly different representation of projection functions of poly¬ 
topes. 

Proposition 3.1. For 1 < k < d — 1, P GV, and E € G{d, k), we have 
Vk{P,E)=u;f\ Y. {E,FfVu{F) ( 'H‘^-^-\du). {i.l) 

FeJ^fc(P) Jn{PF) 

Proposition [ST] is a special case of Proposition 4.1 in m For completeness, 
we include a proof. 
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Proof. For v € \ {0}, put Ef' := E + v. Then, for ^ ^-almost all 

V S E-^ n 5'^“^, we have 


Vk 


{P,E)= Y. MF\E) 


Ar(P,F)nB^ 7^0 


see formula (39) in [1^1 p. 292]. Integration with respect to v € S'^ ^ ’^(E^) 
yields 

^d—k'^k F/) 


= E 


i{iV(p,F^)n£;+ y^0}K£;,F)|Ffe(i^) .(i.2) 


Now assume |(Fi, F')! 0. For fixed F and E, let tt : lin7V(P, F) —)► E^ be the 

orthogonal projection onto E^ with Jacobian Jtt{x) = KP-*-, P'*')| = |(P, P)|. 
Then 


\{E,F)\ 


ln{P,F) 


me 


_L||fc-d -^d-l-k 


"{du) 


= {d—k) f J7r(x)||a;/||x|| I ^{dx) 

Jn{P,F)CB’^ 

= {d-k) [ l{7V(P,F)nB'^nP+7^0 }||u7||u*|| 

Je^ 

where v* is defined by {?;*} = P+ niV(P, F). Introducing polar coordinates, we 
obtain 


|(P,P)| / 

Jn{P,F) 


= {d-k) [ r l{N{P,F)nBME+^mv*/M\\\E- 

Jsd-i-krE-L) Jo 


'J_ II k—a 


X r 


HE^) 


~^{dv) 


= l{N{P,F)nE+:^9}M-^-\dv). 

J gd-i-kf^E^) 

Equation now follows from (13.21) and (13.31) . 


(3.3) 


□ 


In order to transform m into equation m, we choose u G n{P,F) and 
extend it to an orthonormal basis ui = u,U 2 , ■ ■ ■, Ud-k of P"*“. It u ^ E, then 

|(p,p)| = |(p^,p7| 

= ui\E^] + ([0, U2] + • • • + [0, Ud-k])\E^) 

= \\u\E^\\n‘^-^-\{[0,U2] + • • • + [0,Ud_fe])|(P^ n^r^)) 

= • \{M r\F-^,u-^ (ieM, 


yielding 

{E,Fy {u^nF^,u^nE^)^ 

\\u\E^d-k ||y|^±||d-/c-2 ’ ) 

since ||n|i5-*-|| 7 ^ 0 by assumption. 

Taking into account the definition ()1.3p . ProDOsition IS.ll thus yields Theorem 

o 

As we mentioned in the introduction, Theorem 1 1.1 1 can be extended to arbi¬ 
trary convex bodies K using approximation by polytopes and a weak compact¬ 
ness argument to define the flag measure Tk{K,-) as a weak limit. In view of 
(j3.4p . the same is true (with the same measure) for the representation in Propo¬ 
sition [3T] More precisely, we get the following result. Here, and subsequently, 
we use the convention 0 • 00 := 0 for the integrand. 

Proposition 3.2. For 1 < k < d — 1 and K G 1C, there is a Borel measure 
Tj.{K, •) on F{d, d — k) such that, for all E G G{d, k), we have 


v.(K. E) = ^ 10 ^ niK.iiu. U)) . (3.5) 

Proof. As we have already explained, we may assume that there is a sequence 
{Pi) of polytopes converging to K, and such that the measures Tk{Pi, •) converge 
weakly, as i —>■ 00 . Let Tk{K, •) be the limit measure. 

In the following, we fix if S G{d, k) and define, for e > 0, the function 


fe{u,U) 


:= LU 


-1 

d—k 


max(£:, ||M|if-*-||)'^“'' ’ 


(u, U) G F{d, d — k). 


This function is continuous on F{d,d — k) and increases, as e decreases. The 
(monotone) limit / := hm£_>o+ fe is finite for all rt ^ if and is given by the 
integrand in p.5l) (multiplied by Combining monotone convergence and 

weak convergence, we thus get 




F(d,d-k) 


Tk{K,d{u,U)) 


= lim [ fe{u,U) Tk{K,d{u,U)) 

£->■0-1- JF{d,d-k) 

= lim lim f f,,{u,U) Tk{Pi,d{u,U)) 

£->■0-1- 1^00 7F(d,d-fc) 

< lim [ f{u,U)Tk{Pi,d{u,U)) 

JF{d,d-k) 

= lim Vk{Pi,E) = Vk{K,E). 

z—foo 


For the reverse inequality, we use the function /g, given by 

fe{u, U) := f{u, U) ■ l{||u| A-^ll > e}, (u, U) G F{d, d-k), 
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which is upper semi-continuous on F{d^d — k). Again by weak convergence, 
using [M Theorem 9.1.5 (v)], we obtain 


CJ 


A: Tk{K,d{u,U)) 


^ J F{d,d-k) ^ 

> f Mu,U) TkiK,diu,U)) 

feiu,U) Tk{Pi,d{u,U)) 

f(u, C/)l{||'u|i;-^|| < e} Tk{Pi, d{u, C/))) 


JF(d,d-k) 

> limsup 


i—>oo J F(d,d—k) 
\imsnp(vk{Pi, E) 

i .—^ 


F{d,d-k) 


=:Ji 


> Vk{K, E) - ec{k, K) 

“^e^0+ Vk{K,E), 

provided we have Ji < ec{k, K), for some constant c{k, K) independent of i. To 
show this, we notice that 


J^=^d\ E {E,FfVk{F) [ \\u\E^f-'^l{\\u\E^\\<e}n'^-^-'^{du) 

F^Fk(Pi) Jn(Pi,F) 

<ud-\ Y. {E,EfVk{E) [ \\u\E^f-'^n'^-^-'^{du), 

ir^-z7. Jn(Pi.F) 


n(Pi,F)nZ(S,e)^0 


where Z{E,e) C S‘^ ^ is the zonal set 

Z{E,e) := {u G 5'^“^ : \\u\E-^\\ < e}. 

As in the proof of Proposition 13.11 we conclude from (13.31) that 


Ji < ^d-k 


/ E 

/sd-i-.(Er) 

n{P^,F)nZ{E,e)^^ 


X Vk{E\E)l{N{P,, F)r^Et ^ 0} 'H'^-^-^{dv). 


The argument in [HI p. 292] shows that the projections Fjif of the faces occur¬ 
ring in the sum have pairwise disjoint relative interiors for ^-almost all 

V G S'^~^~^{E^). Hence 

U {F\E))<ec{k,K), 

P'^^k(Pi) 

n(Pi,F)nZ(E,e)?S0 

where the final inequality is a consequence of the following lemma. □ 
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Lemma 3.3. Let K G 1C, k G {1, ..., d—1}, L G G{d, k) and e G (0,1). Assume 
that X GhdK has an exterior unit normal u satisfying ||m|L|| > \/l — . Then 

x\L G relbd(iir|L) + eD{K)BL, where D{K) is the diameter of K and Bl is 
the unit ball in L. 

Proof. Let x GhdK with an exterior unit normal u such that ||u|L|| > \/l — e^. 
We may assume that a; = 0. Then v := (rt|L)/||u|L|| satisfies {u,v) > y/l — e^. 
Choose some y GhdK with exterior unit normal v and put g := lin{ri}. Hence, 
g C L, X = x\g = x\L € K\L and y\g ^ relint(itr|L). Thus there is some 
-2^ G [x\g, y\g\ fl relbd(i4r|L), and therefore ||a;|L — z\\ < ||a ;|(7 — y\g\\. 

Define E := lin {m, u}. Then x\E and y\E are points in the relative boundary 
oi K\E with exterior unit normals u and v, respectively. The convexity oi K\E 
and {u, v) > \/l — then imply that \\x\g — y\g\\ < e||a;|i? — y\E\\ < e\\x — y\\, 
which yields the assertion. □ 

Proposition 13.21 shows, in particular, that for each E G G{d,k) the function 
/ : {u,U) I—>■ ||^|^U||i-fe , which may be unbounded, is integrable with respect 
to Tk{K,-). Using (13.411 in (13.511 . we obtain an extension of Theorem 11.11 to 
arbitrary bodies K. 

4 Proof of Theorem 11.21 

In this section we present a proof of Theorem 11.21 (in a slightly more general 
version) using two substantial ingredients. The first is an integral formula 
for the projection function ('Lemma 14.111 proved in |llj : we present the proof 
here for completeness. The second is an integral formula on the Grassmannian 
(Lemma 14.211 , which we derive by using a technique from |12| . This approach 
is different from the one presented in [^, but we will use some techniques from 
in the next section to produce an explicit solution of the relevant integral 
equation. 

The following lemma gives the analog of (lOl . for projection functions. 
Lemma 4.1. Given K G 1C, 1 < k < d — 1 and E G G{d, k), we have 

Vu{K\E) = ^ / \\u\E^f-‘^ 

^d—k J nor K 

X ^ Kic(K-,x,u){Aj{K;x,u),E)^'H‘^~^{d{x,u)). 

|/|=fc 

Proof. The projection function can be expressed as a mixed volume of K with 
the unit ball B^ in E^ and therefore as a mixed functional from translative 
integral geometry, 

ydK\E) = {^^n-\V{K[k],B^^[d-k]) = n-^_,vt%{K,B^^), 

see [T71 (5.68)] and [TSl p. 242]. For the latter, we can use [HI Eq. (7)]. Notice 
that if y is an interior point of B^ and v G E D then {y,v) G nor B^ 
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and there are d — k zero principal curvatures at iy,v), say ki,... ,kci-k, with 
principal directions in E-^, and the remaining k — 1 principal curvatures are 
infinite and their principal direction lie in E. Hence, if Jq = {d — fc + 1,..., d} 
then ;y,v) = 1 and ■y,v) + (v) = E, and 'Kj{B^^ \y,v) = 0 

for all other index sets J of cardinality k. Hence, m Eq. (7)] yields 

Vk{K\E) = -^[ f [ Fk,d-kiAu,v)) 

f^d-k JnoiK Jb^-^ Js>=-^iE) 

X 'Kjc[K\x,u){Ai{K]x,u),E)'^ 

\I\=k 

X n^-\dv)n^-^{dy)n^-\d{x,u)) 

' nor K I 


f x,u){Aj{K-,x,u), E)^ 

^nor K \ T\ 7- 


\i\=k 

X f Fky-k{A{u,v))n^~^{dv)n'^~^{d{x,u)), 

ds'“-i(£;) 


where 


Fk,d-k{0) = 


1 0 rWsmt9V~^~^ /'sm{l-t)0'' 


ojd sin 9 Jq \ sin 9 


sind 


dt 


for 9 S [0,7r). The proof is finished by applying equation (4.1) in [15] showing 
that 

[ Fk,d-kiAiu,v))n>^-\dv) = 
ds'=-i(£;) ^d-k 

□ 

The second ingredient we need is the following integral geometric lemma on 
the Grassmannian. 

Lemma 4.2. Given 0 < k < d and A G G{d,k), there exists a continuous 
function A-A ■ F!{d, fc) —>■ K. such that 


[ ip%AiU){U,B?4{dU) = {A,Bf 
JG(dB) 


lG(d,k) 
for any B G G{d, k). 

Proof. In [H Lemma 4], it is shown that there exist real constants d^’J, 0 < 
p,q < k A {d — k), such that for any A,B G G{d, k) and 0 < p < k A {d — k), 

kA{d—k) 

/ {AU)l{U,Bf4idU)= (4.1) 

JG(d,k) g_Q 

Set 


Pk 


kA{d—k) 

,a{U)~ Y. MAU)l, UGG{d,k), 

P—0 
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for some constants ap which will be specified below. Using 611), we obtain 

kA{d—k) kA{d—k) 

/ iplAU){U,Bf4idU)= E 

JG(d,k) 

Since the matrix D = is regular (see [HI Proposition 1]), we can 

choose 

(cTo, ■ • ■ , ^kA{d—k)^ (f, 0, ■ • ■ , 0)U 

and obtain 

/ ^t,AmU, Bf 4{dU) = {A, B)l = {A, B)\ 

JG{d,k) 

completing the proof. □ 

We can now proceed with the proof of Theorem 11.21 The idea is to apply 
the integral representation (12.51) for the flag measure tpkiK,-) with a function 
g = g{E, ■) arising from Lemma [4.21 To be more precise, let E £ G(d, k) and 
u £ S‘^~^ \ E. Then E^ D «■*■ £ G{u-^, d — 1 — k) and we can apply Lemma IT^ 
in u-^ to obtain a function such that 

/ ^ti-k-E^nuAW){W,Bfvti-k{dW) = {E^Au^,Bf 

JG{u-^ ,d—l — k) 

for all B £ d — 1 — k). In particular, for B = Ajc with \I\ = k (we omit 

the argument and identify the Grassmannian and the oriented Grassmannian), 
we get 

/ ^ti-k;E^nu^ n u^){W, Aif {dW) 

JG{u^,k) 

= [ ‘Pd-i-k-E^nu^i’^'^ '^k^ (dW) 

JG(u^,k) 

= f y^f-i-k;E^nuAU){U,Ai.futi-kidU) 

JG{u^,d-l-k) 

= {E^ n ,Aic)‘^. 

For (m, U) £ E{d,d— k) we define 


(4.2) 


g{E, w, U) := | ^dWl^lE^W^ ‘‘^^^d-i-k-,E^nu^ ^ ^ 

’ ’ 1 0, u € E. 

Then we have, using equations (1^ . 63), 63), 63 and Lemma l4. II 

J giE,u,U) 'ipkiK,d{u,U)) 

= / E ^i4K;x,u) f g{E,u,W-^) 

J nor K I ri ; J G(u-^ ,k) 


(4.3) 


|/|=fe 


X {W,Ai{K;x,u))^ (dW)'H^ ^{d{x,u)) 
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- f Kja(K; X, m)1{m ^ E} 

^d—k Jnor K |/|—^ 

JG(u^,k) 

\ 2 , ( jTTr\ n fd—l I 


X {W,Ai{K-,x,u)Yvl {dW)W-\d{x,u)) 

1 r 


GId—k Jnor K 


^ic{K-,x,u)\{u ^ E}\\u\E 


'-LII k—d-\-2 


\I\=k 

X {E-^ n u'^,Aic[K;x, u)Y 'H‘^~^{d{x, u)) 

1 r 


Gld—k Jnor K 


y^ K/c(iir; X, w)1{m ^ i?}||M|i<^ 


'_L II fc—a 


|/|=fc 

X {E,Ai{K;x,u)Y'H'^~^{d{x,u)) 

= Vk{K\E), 


where we applied the identity 

{E, Aif = {E^ n Ai.Y\\u\eY\^ 

in the second to last step and Lemma 14.11 in the final step. Note that we can 
omit the indicator function l{it ^ E}, since u G E implies that {E,AjY = 0 
and 0 • oo = 0. Thus we obtain (ESI) provided the integrals exist. 

The problem is that the continuous function in Lemma 14.21 will have 
negative parts, in general (see Theorem 15.11 for an explicit formula for the 
coefficients ap). Since the projection factor in g is not bounded, for /c < d — 2, 
the integrals in the above calculation need not exist (an example of this kind 
is constructed in in the related case of mixed volumes). The following 
theorem collects the cases where we have a positive result. Theorem 11.21 follows 
from part (b). 

Theorem 4.3. The assertion of Theorem \1.2\ holds for all K and E, if k G 
{d— 1, d— 2}. For the other values of k and given K G JC, formula (11.61) is valid 

(a) for pK and E G G{d, k), for v-almost all p G 0{d), 

(b) for K and vf.-almost all E, 

(c) for K and all E if Sk{K, •) <C with bounded density, 

(d) for K G V and E G G{d,k), if K and E are in general relative position, 
i.e. L{E) OE^ = {0}, for all E G Fk{K). 

Proof. The cases k = d — 1 and k = d — 2 are covered by the argument given 
before the theorem. Hence, we now assume k < d — 2. 

For e > 0, we define an e-approximation vl{K, E) of Vk{K, E) by 

vl{K,E) / l{|h|i?^|| > e}\\u\EY\’^-‘^ 

^d—k JnoiK 

X Kic[K;x,u){Ai{K;x,u), EY'H'^~^{d{x,u)). 

\i\=k 
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Then, monotone convergence implies that 

v%{K,E) Z'Vk{K,E), as e —>■ 0. 
Hence we have to show that 


lF(d,d-k) 


> e}g{E,u,U)^l)k{K,d{u,U)) 


(4.4) 




IF{d,d-k) 


g{E,u,U) ipkiK,diu,U)) as e —>■ 0 


in each of the cases (a)-(d), with g{E,u,U) given by (14.31) . Obviously, the 
function can be bounded by a constant c > 0. 

(a) In order to apply the dominated convergence theorem in (14.41) we show that 


lF(d,d-k) 


u\EZr-^^^MpK,diu,U)) 




\\u\E 


_L II k—d-\-2 


Sk{pK,du) < 00, 


for zy-almost all p G 0{d). This will follow from 

[ [ \\u\E^f-‘^+^ Sk{pK, du) v{dp) < oo. 

Jo(d) 


'O(d) 

To see the latter, note that 


/0(d) 


[ ||u|i;-^||'" SkipK,du)v{dp) 

= [ [ \\pu\E^t-‘^+^ SkiK,du)Fidp) 

Jo{d) JS<‘-^ 

= ^Mk)I \\v\E^t-<^+^n'^-\dv) 

( k ) 

= [ [ [ \\{cose Ui,sine U 2 )\E 

[ Jo 


_L II A;—<i+2 


(E) . 

X (cos6»)'=-i(sin den‘^-^-'^{du 2 ) n^-\dui) 

■7r/2 


^d—k 


= j^Vk{K)cdkCdd-k / (sin(cos(sin0) 

( fc ) ->0 

= ^dIu Vk{K)uJkUJd-k f {cos sin 9 d9 
{ k ) “'o 

< 00 . 


\d—l—k 


d9 
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(b) From (a) we obtain, for ^-almost all p £ 0{d), 

Vk{K, pE) = Vkip~^K,E) 

= [ \\u\E^r-‘^+^ipt-^^^AU)Mp-^K,d{u,U)) 

JF{d,d-k) 

= [ \\p-A\EA\’^-‘^+AtA\(p-^u)Ap-^u)MK,d{u,u)) 

JF(d,d-k) 

= [ \\u\ipE)A\'^-‘^+AttE)nu^(^)MK,d{u,U)), 

JF(d,d-k) " ’ 

where we used the covariance properties of A and Replacing pE by 

E', we obtain the assertion for i^^-almost all E' £ G{d, k). 

(c) follows from the argument used in the proof of (a). 

(d) For a polytope K we have 

Su{K, •) = ('^ ^ ^) E VAF)'H‘^-^-An{K, F) n •)• 

^ ^ F&Fk{K) 

Hence the support of Sk{K, •) is contained in 

U <K,F). 

F&Fk(K) 

Since L{F) fl E-^ = {0} for all F S Tk{K), we get L{F)-^ r\ E = {0} and 
therefore n{K, F) C\ E = %. A compactness argument then shows that there is 
some e > 0 such that ||u|£’-*-|| > e for all u in the support of Sk{K, •). □ 


5 Determination of a concrete solution 


The proof of Theorem 11.21 given in the previous section shows that a continuous 
function g exists which yields the integral representation (11.61) . In the following, 
we give an explicit expression for this function. This will follow from the proof 
of Lemma l4.2l if we have an explicit solution a := (oq, ..., o:kA{d-k )) of the linear 
system 

a-D = (l,0,...,0) (5.1) 

with coefficient matrix D = ■ The following theorem gives such a 

solution. 


Theorem 5.1. An explicit solution a of (15.11) is given by 




it)m 

ik+i)ar 


m = 0,..., /c A (d — fc). 
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In order to prepare the proof of Theorem l5.ll we recall from [TH p. 649] the 
equation 


kA(d—k) 

XI 4,jF{d,k,j,Tn,i) 
j=o 

where 


k 

F{d,k,j,ni,i) := X 
1=0 



(5.2) 


d — k — i 
j + I + m — k 


and where the constants cf for 0 < j < k A {d — k), are defined as follows. 
Let ei,..., Cd be the canonical basis of and let E G G{d, k) be the subspace 
spanned by ei,..., and F G G(d, k) the subspace spanned by ei,..., ek-j, 
Cfc+i,.. .,ek+j. Then 


cij= [ {E,Vf{F,Vf4{dV) 

JG(d,k) 

depends only on the dimensions d,k,j (and not on the particular choice of the 
bases), as follows from [TH Lemma 3] and the explanation on p. 647 of |12) . 

In the next two lemmas it is convenient to use calculations in the Grassmann 
algebra /\K‘^ of R.'^. As in [12], we introduce scalar product (and a norm) on 
/\k which are induced by the scalar product of and we also use the same 
notation. 

Lemma 5.2. For any nonnegative integers i, fc, d such that 0 < i < k and 
2k < d we have 

„d. _ rd d -1 

^k,i ^k ^k — l,i'> 

where 

d _ k{k + 2 ) 

“ d(d + 2)' 

Proof. Let E, E be defined as above and let V be another A:-subspace. If V 
is not orthogonal to ei, then V has an orthonormal basis vi,... ,Vk such that 
vi,... ,Vk-i are orthogonal to e := ei and Vk is the normalized orthogonal 
projection of ei to 14. Then we have 

(A, Vy = \\ek+i A • • • A ed A ■ui A • • • A VkW^ 

= \{vk,e)\‘^\\ek+i A-- - AcdAviA-- - A Vk-i\\‘^ 

= \\e\V\\^{EC^e^,Vf^e^)‘^ 


and a similar relation for F. Since 


cL = 


IG(d,k) 


{E,vy{F,vyuUdv) 
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the above identity yields 


cU = [ \\e\V\\HE ne^,Vn e^f{F He^^Vn vi{dV). 

JG{d,k) 

We now apply a special case of Lemma 4.1 in [T3] (see also [TH Theorem 7.2.6]) 
which yields 


lG{d,k) 


h{v)4{dv)= 


27 r 2 r (I) JG(e-L./c-l) 


(5.3) 


X / h{W + {v))\{e,v)\'^-^n^-'^{dv),.ti{dW), 

yS<i-fc(Vy-L) 

whenever h is an integrable function on G{d, k). For our choice of h we obtain 

r(i) r r 




27r^r (I) JGie-^.k-l) 

X {Ene-^,Wf{Fne-^,Wfiyf_i{dW) 

r(f) f 


\(v,e)nv,e)r^H‘^-^{dv) 


\{v,e)r--^n^-'^{dv)cizi. 


2tt 2 r (I) Js‘^-'‘(w-^) 


r{l) 


fc—1,2 


27r^r(|) r(^) 

_ fe(fc + 2) 

“ d{d+2) 

yielding the assertion. Here, we have used the first equality from Lemma 17.11 in 
the last but one step. □ 


Lemma 5.3. For any positive integers k,d such that 2k < d we have 


„d _ jd i 

k — ^k 




where 

= 

did+ 2)' 

Proof. Let E, F be the subspaces as defined above. Let V be another fc-subspace 
which is not orthogonal to e = ei, let ..., Wfc be the orthogonal basis of V as 
in the proof of Lemmaand let m be a unit vector from F fl (F fl e-*-)-*-. We 
set 

ic := ei A • • • A Cfe A e2fc+i A • • • A A ui A • • • A Vk-i 

and choose a unit vector a orthogonal to the vectors involved in the definition 
of w. Then 


2k d d 

= '^{ej,af\\w Aaf = ||w A af = ||wf. 

j=k+l j=l j=l 
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Since {F fl ef, V fl e-*-)^ = ||w A ej\\^, we obain 


{F, V)'^ = \\ei A • • • A Cfc A e 2 k+i A • • • A A wi A • • • A VkW^ = |(u,Wfc)p||w|| 


2k 


= \{u,Vk)f {Fnef,Vne^f. 

j=k+l 


Hence, 


2k 

cik= [ \{e,Vk)\‘^\{u,Vk)\‘^ ^ (i;ne-^,Hne-^)^(Fne^,Hne-^)V^((iH). 

JG(d.k) 


IG{d,k) j=k+l 

Using (15.31) again, we get 

r(f) / f 


^k.k 


2Tr~r (I) iG(e^,fc-l) Js'i-’‘(W^) 




2k 


X ^ {Ene^,Wf{Fnef,W^)^i2-,_,{dW) 
j=k+l 

rM) , f 


2tt 2 r (I) 

The second equality from Lemma l7.II yields 




[ \{e,v)\^^^\{u,v)\^n'^ ^{dv) = 2'K'' 2 


hence we have 


^k,k 


d{d + 2) 


rm’ 


^d-l 


□ 


Corollary 5.4. For integers 0 < fc < d — 1 and 0 < i < k A {d — k), we have 

Proof. For 2k < d, we can iterate the recursive formulas from Lemma |5.21 and 
Lemma 15.31 and get 

jd jd—1 jd—k-\-i-\-l jd—k-\-i jd—k-\-i—l -rd—k-f-l 
di Ji_i ' ' ’ ? 


a _ ja ja-i _ r 
^k,i ^k^k — 1 


2+1 


which yields (15.41) . 
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For 2k > d, we observe that the right-hand side of (15.41) does not change if 
we replace k hy d — k. Hence, we only need to show that cf ^ j holds 

(for 2k > d and 0 < i < {d— k)). Let E,F G G{d, k) have a (fc — z(-dimensional 
intersection. Then, 


cU= [ {E,Vf{F,Vf4idV) 

JG(d,k) 


IG{d,d-k) 


{E,v^y{F,v^yi.Uidv) 


IG{d,d-k) 


{E^,vy{F^,vyvi_ydv) 


= c. 


— k,i^ 


since E-^ D has dimension d — k — i. 


□ 


Using (15.21) . (15.41) and the formula for am, we obtain 

E ,d,k 
^rndmy 

m 

-f 1\ ^ (d-I-2 — fc)(fc-f 2) 




m J (d-f 2)(j-|-l)(j-|-2) 


F{d,k,m,j,i) 


{d + 2 — k){k + 2) 
2(d-f 2) 




d -I- 1 


E 


J + 2 


F{d,k,m,j,i)- 


We replace m hy k — m and j hy k — j. Then, we get from Theorem 17.21 


jd,k {d + 2 — k){k + 2) . , 

E — E(-1) 


k—m 


X 

J 

k — i 
k 


2{d + 2) 

E 


d -f 1 
k — m 


k-j + 2 
2 


F{d, k,k — m^k — j, i) 


which is 1 for z = 0 and 0 otherwise. This completes the proof of Theorem 15.II 


6 Final discussion 

We add a few remarks concerning the functional analytic aspects of the two 
integral representations which we obtained for the projection functions. Let 
C{F{d, d — k)) denote the Banach space of continuous functions on F{d, d—k). 
We consider the integral transform Tk ■ C{F{d, d — k)) —>• C{F{d, d — k)), given 

by 

{Tuh){u,L)-.= [ {L,Myh{u,M)v'y}ydM), 

J G{{u) ,d—k) 
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for {u,L) G F{d,d — k). The transform Tk is obviously continuous and self- 
adjoint, therefore it can be extended to a weakly continuous linear map on 
the dual space of finite signed measures on F{d,d — k). It follows from the 
definitions (IE3 and (lESl) that, for a polytope P, the measure tpk{P, ■) is the 
image of rfe(P, •) under Tk- This also holds for convex bodies K G /C, if we choose 
Tk{K, •) as the limit of a converging sequence Tk{Pi, •), as we did in Section 3. 
The non-uniqueness of this limit implies that the linear map Tk is not injective, 
as was already remarked in m- 

Using this mapping, we could deduce Theorem 11.21 for polytopes P from 
Theorem ll.il (up to a constant), if we find a function g satisfying the integral 
equation 


/ G{{u) ,d—k) 


{U,F^Yg(E, 


,U)v‘Y}k{dU) = 


{u^ n F^,u^ n e^Y 


\\u\eyY 


-k-2 


for E,F G G{d, k) and uEE. For a discussion of the latter, we may fix u G 5"^“^ 
and replace by K" (n = d—1). Instead of integrating over all U G G{{u),d — 
k), we may then integrate over all L G G(n,n — k) and seek a function /e on 
G{n, n — k) such that 


/ {L,EYfE{L)i2n-k{dL) = {E,FY (6.1) 

G{n,n—k) 

holds for all E,E G G{n,n — k). As we have seen in Lemma 14.21 (16.11) has 
a solution, for each E G G{n,n — k), given by a continuous function Je on 
G{n, n — k). Since E here is an arbitrary subspace, we may choose E to be the 
linear hull of the vectors ei,..., Cn-k in tbe standard basis ei,..., e„ of K." and 
consider the integral operator Sk ■ C{G{n, n—k))^ C(G(n, n — k)), defined by 

Skih){E):= [ {L,EYh{L)En-kidL), 

J G{n,n—k) 

for E G G(n,n — k),h G G{G{n,n — k)). The mapping Sk is a slight variant 
of the mapping Tk mentioned above. Again, Sk is self-adjoint, but not injec¬ 
tive. However, if we consider the subspace £ C C{G{n,n — k)) considered in 
[12], then Sk maps C to C and, when restricted to £, it is a bijection by [121 
Corollary 1]. To recall the definition of £, we define A/ := lin {e^ : * G /}, where 
/ C {l,...,n}. Then, £ is the (^"^)-dimensional linear space of continuous 
functions on G(n,n — k) spanned by the functions {Ej,-Y with |J| = n — k. 
Since for the above choice of E, we trivially have (A, -Y G £, the function /e 
can be defined as the inverse image of {E, - Y in £. Actually, this is the approach 
we used in the proof of Theorem 11.21 The operator Sk is the special case a = 2 
of the a-cosine transform, which has gained recent attention (see, for example, 
[T] and the literature cited there). 

We also add some comments on the flag formula dug for the projection 
function Vk{K,-)- First, the special case k = d—1 yields (|l.lll again. Namely, 
(EU shows that ipd-iiK, •) = sj^liiK, •) and the latter equals Sd-i{K, •) under 
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the natural identification u O (it, {0}). Then, the function g{E,u,U), with 
the projection hyperplane E and U = (it), must equal ^|(ii,i;)| (up to a linear 
function), where v is the normal of E. The corresponding question as to whether 
(fra implies (EH) if is a generalized zonoid seems to be more complicated. 
Although there are formulas expressing the flag measure tpkiK, •) of a generalized 
zonoid K in terms of the generating measure p{K, ■) of K (see [TH Corollary 3]), 
a possible connection between (ira and (EH) is complicated by the fact that a 
signed measure 4> on G(d, k) satisfying 

VkiK,E) = ^ [ \{E,FmdF), EGGid,k), 

is not uniquely determined, if fc S {2,..., d — 2} (see [4]). 

Using m, we can formulate the projection formula (11.61) also with •) 

replaced by Namely, let g be defined by 

g{E, It, V) := — —g{E, it, U + (it)), E S G(d, k),{u,V) € F^{d, d — 1 — k). 

Wfc+l 

Then we have 

Vk{K,E)= [ ~g{E,u,V)S\^-’^-^\K,d{u,V)) (6.2) 

J F-^{d,d-l-k) 

for K £ K. and almost all E £ G{d, k). Formula (16.21) can easily be generalized 
to lower order projection functions 

v,kiK,E):=V,{K\E), E£G{d,k), 

1 = 0 ,..., fc, and to mixed projection functions 

v{Ku...,Kk,E) :=V^^HKi\E,...,Kk\E), As G(d,fc), 

(where {Mi, ... Mk) denotes the mixed volume (in E) of convex bodies 
Ml, ..., Mk C E), since both sides of (16.21) allow a polynomial expansion for K+ 
rB’^, r > 0, respectively a multinomial expansion for ri ATi + • • • + rmKm, Ki S 
> 0. Here, for the right-hand side. Proposition 2 and Theorem 6 in m 
can be used, the latter introducing mixed flag measures. Explicit formulas are 
given in [3] and ^0] . 

Since projection functions in general do not determine the shape of a non- 
symmetric convex body, one may also ask for flag representations of directed 
projection functions, as they were introduced and studied in laiT]. We leave 
this as a question for further investigations. 

7 Appendix 

The following lemma was used in Section 5. 
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Lemma 7.1. If d> 2, u is a fixed unit vector in and p > —1 then 
f \{u,v)\^n‘^-\dv) = 

If further q > —1 and w is another unit vector perpendicular to u, we have 

P p / P+1 \ p / Q + 1 \ 

/ \{u,v)\P\{w,v)\'^n‘^-^idv) =2Tr^ ^ 

Jgd-l p / d+p+q \ 

Proof. We apply the coarea formula for the mapping g v ^ ('a,v)^ defined on 
the unit sphere. Since 


Jig{v) = 2{u,v)\Jl - (u,u)2, uG+ \ 

and since for 0 < r < 1, g~^{r} is consists of two spheres of dimension d — 2 
and radius y'l — r, we get 


\{u,v)\P'H‘^ ^{dv)= I = +(1 - r)“ 2 *a;d_ 2 dr 


nl 1 

1 V-1 


ISd-1 


pi ^ (13 

/ r^ {1 — r)^~ dr uJd -2 

r(^)r(^) 2+/^ 

r(^) r(^) 


and the first assertion follows. 

For the second formula we again use the coarea formula with the mapping 
g and get 


ISd~i 


\{u,v)\P\{w,v)\‘^ n‘^ ^(dv) 


[ [ \{w,v)\‘‘n^-^{dv)dr. 

Jo ^ J 


1 T 

1 p—1 

-r ^ (i — r) 2 

/O ^ 

The two mappings 

z !->■ i-^ru + Vl — rz, z G 

map injectively onto g~^{r}, and the area formula yields 

[ \{w,v)\‘^n{dv) = {l-r)^ [ \{w,z)\‘^n‘^-^dz) 

d-2+, d_2 r( 2 ±i) 

= (1 - r)^~27r~ ^ / 


(«±i) 
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by the first (already proved) equality. The final result follows now from the 
standard integral 


r ^2 (1 _ r)' 


tj —3 + g 


dr = 



□ 


Next, we give a proof of the combinatorial identity which was used in Section 
5. As usual we define, for o G Z and z G C, 



z{z — 1) • • • (z — a + 1) . 


if a > 1, 


:= 1 and ( ) := 0 if a < 0. 

<CLj 


Theorem 7.2. Let d,k,i G Nq and d> k — 1. Then 


m=0 ^ ^ 

/ fc — i \ /fc — A / d — 2k + i \ 
^ h'n Vv \m - i) \j - e) \k + e - m - j) 



(7.1) 


Proof. The proof is divided into several steps. 

(1) For fixed d,k gNq and d> k—1, both sides of (17.11) are polynomials in i 
of degree k at the most. Therefore it is sufficient to verify (17.11) for i = 0,..., fc. 
Thus the following has to be shown: 

For d, A: G No with d > A: — 1 and i G {0,..., A:}, 


m=0 ^ i=0 ^ ^ 

/A / A: — i \/A; — / d—2k + i \ 


(7.2) 


(2) Let A: G No and i G {0,..., A:} be fixed. Then the left-hand side of (17.21) 
is a rational function g in d of degree less or equal 0 with possible poles at 
—2 ,... ,k — 3 (the singularity at d = A; — 2 is removable). All poles are simple. 
Hence it is sufficient to check that 


i'esd=u{q) = 0 for u G {-2,..., A: - 3}, (7.3) 

lim q{d) = S^k- (7.4) 

|d|—>^oo 
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Since 




/ d—2k-\-i \ 


u — 2k + i \ fk — m 


(- 1 ) 


k —771 — 2 —z 


' {d +2) \k + £-m-jJ \ u + 2 

whenever u £ Z and m£{0,...,fc}, condition (17.3|) is equivalent to 

k — i\fk — i\f u — 2k + i 




'jk—m 
m—0 j—0 


^^ + 2- j\ fi 


E 


Moreover, dH follows as soon as 


k + 2 




fk + 2-J 


k — m 
W “h 2 

-1 


m — tj \j - 

k—7n—2— 


{-ly 


k + i — m — j y 
“ = 0. (7.5) 


m—0 


j,l=0 

3=^ 


k — i \ fk — i 


m — ij \j — i 


= S. 


ik 


(7.6) 


has been verified. 

(3) First, we establish (17.61) . For the left-hand side of (17.61) . we obtain 

k 


k + 2 


m—0 



■T. 

j=o 


k + 2 fk + 2 — j 


k — i 
m — j 


■E 

j=0 





k-j 




m=0 


m 


k + 2^ (k + 2- j 


■E 

3=0 





; )(_i)fe-i(i_ i)fc-*. 


(7.7) 


where we used that, in case i > j, the second sum is extended from 0 to A: — i 
effectively. Now (17.6|) clearly follows if i < fc. If i = fc, then (17.71) equals 


k + 2 fk + 2 — j 

9 2^ I 9 


3=0 


(- 1 ) 


k-j 


k + 2 




k + 2 




2 \ k J 
k + 2 fk + 2\ 


(- 1 ) 


k-j 


j=0 


( J I 


fk + 2-l 


{-ly 


= 1, 
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where 


B-iyQ "(-i)"!"/). ^eC.teNo, 

was used (this can be proved by induction with respect to k). 

(4) We turn to the proof of (17.51) . Let k G No, i G {0, ...,fc} and u G 
{—2,..., fc — 3}. Replacing m — £ by m, we see that (17.5p is equivalent to 

■^/fc + 2 —^ fi\fk —i\fk —i\fu —2k + i\fk —m —t\ 

SI 2 )=»■ 

(7.8) 

Note that (*_|_ 2 ) is a polynomial in z of degree m + 2 and (^) is a polynomial in 
z of degree w, where 0 < w < u + 2. Hence is a linear combination of the 
polynomials (^), re = 0,..., u + 2. 

(5) The preceding discussion demonstrates that (17.81) follows from the stronger 
assertion: For k G Nq, i £ {0,... ,k}, u G {—2,..., fc —3} and w G {0,..., u + 2}, 

/fc + 2— A ^ /A/fc — A/fc — A/u — 2A: + A/m + 

j—0 ^ ' £—0m—0 ^ ^ / \J / \ J/ \ 


(7.9) 

Using the relation (Vandermonde convolution) 




we conclude that 


a 


= E E 

Wi-\-W2—W 


k — i 
j - 


W2 


E 

\m=0 


k — i\ f u — 2k + i 
k — m — j 


The expressions in brackets can be simplified, since we have, for H > 5 > 0 and 

C,D£C, 



A 


E 

a—B 


C---{C -a + l)D---{D- A + a + l)a! 
a\{A- a)\B\{a- B)\ 


C • • • (C - 5 + 1) 

m 




(C-B + D\ 

[b)[ A-B J’ 
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which holds in fact for all A,B>0 and C,D G C. Thus 

i \ — W 2 + k — i\fk — i\fk — i — wi+u — 2k + i 


a 


= E 


^2/ \ J — '^2 


101 + 102 = 1 ^ 

The left-hand side of (|7.9I) now simplifies to 


Wl 


k- j -wi 


/ i \ /k — i\ /k + 2 — j\ ^ fk — W 2 \ fu — k — wi 

V y ^ V 2 ) \j-w2)\k-j-w^ 


= E 


k — Wi —W2 
X 


k- 

W 2 / \ Wl 


E 

3=0 


k — W2 \ (k + 2 — W2 — j 

1 


-\ -1 


u — k — Wl 

k — Wl — W2 — j J 


We show that P = 0. Using the relation 

u-k-wi \ ^ A-Wl - W 2 - J - 1 - {u^k-wi) 

yk — Wl — W2 — j J \ k — Wl — W2 — j 

k—U — W2—j — 
k — Wl — W2 — j 
t—U — W2—j- 
k — u + Wl — \ 

we get 

k — Wi —W2 




= (-l) 
= (-l) 


(2k-u-W2- 3-1 


= ik-_W2\^^_^y(k + 2-W2-3'^ 

7=0 


= ^E ^ (^‘2‘k-u-W2-3 

j = 0 V- 


k — u + Wl — 1 J 

— U — W’, 

k — u + Wl — 1 


—lU) 

The last equality follows from the fact that for j > k — wi — W 2 , we also have 
2fc — u — W2 — 1— j<fc — w-fwi — 1 

so that the third binomial coefficient is zero. The denominator of 7 has simple 
zeros at fc -I- 2 — u' 2 , fc -I- 1 — W 2 . Moreover, the numerator of 7 has zeros at 
2fc — u — r (;2 — 1,..., fc — 1112 — Wl -I- 1 and 


2k — u — W 2 — l>k + 2 — W2>k+1 — W 2 >k — W 2 — wi + 1. 
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Hence, 7 is a polynomial in j of degree at most k — u + wi — 3. The degree of 
7 can be estimated by 

k — u + wi —3< k — u + w — W 2 — 3< k — u + u + 2 — 3 = k — 1. 


By m (5-42)], 



for any p & C[x\ with deg(p) < A — 1, hence /? = 0 and therefore a = 0. This 


establishes (EH). 


□ 
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